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Abstract 

Models  of  complex  systems  can  be  differentiated  by  their  ability  to  reproduce  or  generate 
system  behavior,  by  their  prediction  power,  by  their  robustness,  or,  conversely,  by  their 
sensitivity  to  inputs  and  parameters;  by  their  uncertainty  (if  captured);  and  by  their 
intelligence.  Even  the  term  “prediction”  is  not  unique.  First,  a  first-principle  (physically 
based)  distributed  parameter  model  could  be  an  excellent  predictor  if  (a)  it  captures  the 
main  system  behavior,  and  (b)  its  parameters  and  inputs  are  known  accurately;  otherwise, 
it  would  fail,  possibly  drastically.  Second,  predictive  power  depends  on  the  data,  on  the 
goal,  and  on  the  time  scale.  For  example,  scheduling  of  pumping  and  injection  in  an 
oilfield  for  maximum  profit  over  the  next  5  years;  or  pumping  from  a  contaminated 
aquifer  in  order  to  maintain  certain  (low)  concentration  at  a  compliance  point  for  the  next 
20  years,  vs.  prediction  of  plume  migration  in  groundwater  towards  a  nearby  river,  over 
time:  in  each  case,  the  model  has  a  slightly  different  expected  function,  as  well  as 
different  intelligence  type.  The  paper  reviews  the  recent  developments  in  subsurface  fluid 
flow  management  such  as  optimization  of  oil  production  and  groundwater  remediation 
(both  sharing  similar  practices,  though  for  different  purposes)  as  a  continuous  struggle  to 
increase  intelligence  by  (a)  adapting  new  tools  such  as  artificial  intelligence  and  dynamic 
stochastic  control;  (b)  attempting  to  integrate  these  tools;  and  (c)  reducing  uncertainty. 
Although  the  systems  discussed  seem  specific  to  the  (mathematical)  geosciences 
(specifically  to  oil  reservoirs  and  contaminated  aquifers),  and  although  these  systems  are 
very  different  from  man-made  machines,  similar  rigid  structure  and  reliance  on 
differential-integral  calculus,  as  well  as  the  serial  processing,  knowledge  evolution,  and 
uncertainty  propagation  from  one  discipline  to  the  next  exist  in  most  science  and 
engineering  fields,  and  so  does  the  need  for  a  paradigm  shift.  Given  the  need  in  adaptive, 
intelligent  control/planning/optimization  of  such  systems,  the  progress  of  these 
segregated  efforts  towards  a  multiresolutional  decision  support  system  is  inevitable, 
highly  desirable,  and  highly  promising.  However,  we  are  still  facing  the  challenge  of 
perfonning  and  defining  optimal  integration  between  PDE  models  and  multiresolutional 
representations,  and  since  such  integration  depends  on  model  quality  and  data  quantity 
and  quality,  this  is  an  adaptive  integration  process  as  well.  One  criterion  for  such 
“optimal”  integration  would  be  uncertainty  reduction  (resulting  from  the  integrated  MR 
system),  which  brings  up  yet  another  challenge:  to  define  the  metrics  for  uncertainty 
reduction. 
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1.  Introduction 

Management  of  distributed-parameter  systems,  particularly  where  complex  natural 
processes  intersect  with  human  industrial  practice  and  theoretical  knowledge,  is 
extremely  difficult  to  analyze  and  optimize.  From  a  knowledge  organization  perspective, 
this  area  of  practical  knowledge  is  in  disarray.  For  example,  knowledge  in  the  petroleum 
industry  relates  to  many  scales  of  representation,  yet  this  fact  is  not  taken  into  account  in 
an  organized  manner. 

Indeed,  the  oil  industry  is  a  live  example  of  the  need  to  integrate  multiple  disciplines 
presently  not  integrated,  including  huge  volumes  of  raw  data  (particularly  in  geophysics) 
and  multiple  computational  and  conceptual  models  of  geology,  geophysics,  and  reservoir 
dynamics.  Although  the  need  in  unification  of  the  bodies  of  knowledge  in  these 
disciplines  and  the  associated  (tremendous)  benefits  of  analysis  enhancement  capability 
has  been  widely  recognized,  progress  in  this  direction  has  been  very  slow.  The  main 
reason  for  this  is  the  rigid,  segregated,  serial  process  of  knowledge  and  model  building  - 
from  geophysical  signals  to  simulation-optimization  of  fluid  dynamics  in  porous  media  - 
a  process  that  has  evolved  naturally  during  the  last  century,  where  the  simulator  has 
become  the  focal  junction  where  all  the  knowledge  and  understanding  of  the  physical 
processes  and  material  properties  are  being  filtered  and  concentrated  in  the  form  of 
partial  differential  equations  (PDE  that  describe  mass  balance  of  oil,  water,  and  gas  in  3D 
space)  whose  coefficients  (or  distributed  parameters)  suppose  to  capture  the  physical- 
chemical  properties  of  the  medium  on  a  particular  scale,  everywhere  in  the  modeled 
subsurface/reservoir  domain,  assuming  some  “known”  boundary  conditions  and  initial 
conditions.  In  the  following,  we  will  use  subsurface  flow  systems,  particularly,  oil 
reservoirs,  as  examples  of  managed,  distributed  parameter  systems. 

Oil  reservoirs  are  complex  systems  on  all  scales.  Decisions  such  as  pumping  and 
injection  (schedule  and  rates),  new  well  placement,  and  (directional)  drilling  in  an  active 
oil  field,  are  typical  of  the  complex  relationships  between  reservoir  characterization  and 
oil  field/reservoir  management.  The  solutions  to  such  problems  involve  a  complex  system 
of  multiphase  flow  equations  (linked  PDE)  in  a  heterogeneous  domain  (reservoir),  as  well 
as  economical  factors  such  as  short-tenn  and  long-tenn  oil  price,  worth  of  information, 
inventory/storage/delivery,  cost  of  drilling,  maintenance,  production,  etc.  Well  drilling 
and  construction  equipment  are  costly  and  cannot  be  afforded  as  frequently  as  necessary, 
while  prevalent  information  gaps  render  decision-making  uncertain  and  hence,  risky. 

Yet,  the  advantages  of  reservoir  simulations  should  not  be  underestimated  either. 
Models  or  simulators  based  on  PDE  solutions  provide  physical  insight  into  various 
important  flow  phenomena,  as  well  as  the  general  behavior  of  the  fluid  movements  in  the 
reservoirs,  under  scarcity  of  spatial  data  typical  of  both  old  and  young  reservoirs, 
particularly  the  latter.  The  ability  to  capture  the  essence  of  the  complex  physics  behind 
the  reservoir  responses  to  pumping  and  injection  is  the  strength  of  the  simulator  and  the 
essence  of  its  intelligence.  However,  this  strength  could  promptly  become  its  weakness 
where  (a)  natural  geological  heterogeneities  on  certain  scales  are  being  missed,  or  (b) 
physical/chemical/thermodynamic  processes  are  being  missed  (e.g.,  leaching 
geochemistry;  instability  of  the  oil-water  interface),  or  (c)  uncaptured  (or  erroneous) 
boundary  conditions,  all  of  which  would  lead  to  wrong  predictions.  In  other  words, 
wherever  an  essential  physical  phenomenon  on  any  scale  is  being  missed,  the  simulator 
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looses  its  intelligence,  i.e.,  its  ability  to  predict  short-term  reservoir  responses  and  long¬ 
term  oil  reserves  and  revenues. 

The  main  question  we  attempt  to  address  in  this  paper  is  how  to  merge  the 
advantages  and  use  the  intelligence  of  existing  models  and  interpretations  in  a 
comprehensive  intelligent  system  that  could  take  advantage  of  such  physically  based 
intelligence,  while  eliminating  its  limitations.  In  order  to  answer  this  question,  we  first 
need  to  understand  the  structure  and  limitations  of  current  approaches  to 
optimization/control  of  subsurface  fluid  flow  and  solute  transport.  The  reader  could 
notice  that  although  the  problems  discussed  here  seem  to  be  specific  to  optimization  of 
oil  production  and  groundwater  remediation,  similar  rigid  structure  and  reliance  on 
differential-integral  calculus  exist  in  most  engineering  fields,  and  so  is  the  need  for  a 
paradigm  shift  when  planning/control/optimization  become  the  focus.  Yet,  we  should 
also  keep  in  mind  some  major  difference  between  the  geosciences  (or  natural  systems  in 
general)  and  man-made  machines.  In  hydrogeology  and  oil  reservoirs,  we  deal  with 
multiphase  flow  in  heterogeneous  formations,  with  transient  flow  and  transport 
phenomena  occurring  on  all  scales,  with  nonlocal  dependency  on  (unknown)  fluid  flow 
everywhere.  However,  whenever  we  focus  on  operations’  scheduling,  feedback  and  feed¬ 
forward,  this  complex  system  becomes  similar  to  other  complex  operations;  finding  a 
new  well  location,  however,  requires  considering  the  complete  heterarchy  of  transient 
flow  phenomena  in  space,  often  with  major  data  gaps.  Such  data  gaps  challenge  all 
models,  and  consequently,  any  integration  of  PDE  models  with  MR  knowledge 
representations  (and/or  MRDS).  Such  integration  seems  to  be  data-dependent,  and 
requires  optimization  on  its  own  merit,  weighting  the  robustness,  prediction/anticipation 
(goal-oriented)  power,  the  uncertainty  associated  with  different  representations,  as  well  as 
uncertainty  reduction  produced  by  such  integration. 


2.  Current  approaches  to  simulation-optimization-control  of 
distributed-parameter,  subsurface  flow  systems 

The  cutting  edge  subsurface  fluid  management  such  as  oil  reservoir  optimization  and 
groundwater  remediation  control  under  uncertainty  has  been  moving  in  three  major 
fronts:  (1)  operations  research  (including  stochastic  models  and  risk  assessment);  (2) 
stochastic-dynamic  control;  and  (3)  artificial  intelligence  (AI),  particularly  artificial 
neural  networks  (ANN),  genetic  algorithms  (GA),  and  fuzzy  logic  (FL).  However,  under 
the  current  structure  of  serial,  segregated,  and  isolated  “disciplines”  that  process  the 
infonnation  from  geophysics/explorations  to  reservoir  characterization,  reservoir 
simulations  (or  flow  and  transport  models  in  hydrogeology),  and  optimization/control,  it 
is  impossible  for  these  three  fronts  to  merge  into  a  unified,  integrated  approach,  nor  could 
a  major  progress  in  oil  field  management  be  made.  Under  the  current  paradigm, 
optimization/planning/control  of  these  complex  systems  have  been  handicapped  by 
uncertainty  on  one  hand,  and  prohibitive  computer  power  &  time  requirements  on  the 
other  hand,  without  benefiting  from  all  available  information. 

The  typical  approach  to  reservoir  characterization  and  management  sketched  in 
Figure  1  shows  the  different  subsystems  that  constitute  both  exploration  and  production. 
The  figure  is  highly  simplified,  with  many  subsystems  not  shown,  such  as  subdivision  of 
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exploration  (remote  sensing,  surface-  and  borehole-geophysics;  seismic,  electrical, 
electromagnetic,  micro-gravity,  SP,  etc);  geologic  investigations  (structural  geology, 
geochemistry,  lithology,  bio-stratigraphy);  subdivision  of  production  (well  pattern  design, 
injection  and  pumping,  gas,  water,  and  displacing  fluids);  and  well  construction,  -  all  of 
which  are  complex,  interdependent,  and  require  real  time  updating  and  decision  making. 
The  figure  illustrates  current  model  construction  and  subsequent  optimization. 

Blocks  1  and  2,  and,  to  some  extent,  Block  3  (conceptual  model),  represent  reservoir 
characterization,  which  plays  a  crucial  role  in  exploration  and  subsequent  reservoir 
management.  Typically,  the  conceptual  model  (Block  3)  of  the  reservoir  is  an  undeclared 
part  of  the  simulator;  this  is  where  all  the  geology  is  filtered,  upscaled,  and  translated  into 
the  simulator’s  parameters,  which  inherently  entails  averaging  and  discarding  of 
information  (acting  as  a  low-band  filter),  including  small  scale  features  that  may  be  crucial 
(in  which  case,  their  large-scale  influences  would  be  modeled  as  different,  large-scale 
parameters  -  e.g.,  dispersion  coefficient).  Most  of  the  assumptions  and  decisions  related  to 
reservoir  representation  are  made  at  the  conceptual  model  stage,  and  are  subject  to 
modeler’s  understanding  and  experience. 
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Figure  1.  A  schematic  view  of  current  approaches  to  reservoir  optimization. 


Block  4  is  the  current  quantitative  “brain”  (or  predictor),  typically  a  distributed- 
parameter,  complex  PDE  solver  that  may  include  several  linked  PDE  with  their  auxiliary 
constitutive  functions  (mostly  determined  in  the  lab,  on  a  lab-scale),  or  less  commonly,  a 
cell  model,  represented  by  Ordinary  Differential  Equations  (ODE),  implying  a  larger 
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scale,  i.e.,  lower  resolution).  Within  this  computational  block,  additional  analytical  fonns 
are  utilized  for  computing  parameters  and  constitutive  relationships,  as  well  as  local 
modeling  (e.g.,  the  Buckley-Leverett  model  of  displacement).  Ideally,  the  simulator 
contains  multiple  analytical  models  functioning  at  different  time  scales,  and  demonstrates 
processes  working  at  different  levels  of  resolution.  However,  current  models  are  far  from 
this  ideal. 

Due  to  the  embedded  upscaling  and  loss  of  information,  feedback  from  the  reservoir 
simulator  (4)  to  the  geological  model  (2)  is  not  reliable,  except  for  special  cases  where 
certain  disparity  exists  between  the  measuring  window  and  the  scale  of  the  geologic 
feature,  and  where  only  a  piece  of  the  puzzle  is  missing  (e.g.,  in  well  testing  with  an 
appropriate  monitoring  system  in  place,  or  in  dual  porosity  systems  where  the  rock 
properties  are  well  characterized  on  all  scales).  Typical  inverse  or  automatic  calibration 
procedures  determine  some  local  reservoir-fluid  interaction  “properties”  that  fit  a 
particular  (and  hence,  uncertain)  model.  Subsequent  interpretations  of  geologic  features 
based  on  inverse  modeling  (or  calibration  of  the  simulator)  are,  therefore,  speculative. 

2.1  Reservoir  Simulations  and  Groundwater  Modeling  Problems 

Historically,  modeling  of  fluids  flow  in  porous  media  using  PDE  started  by  Muskat 
in  the  1930’s  [52,  53].  Until  that  point,  predictions  of  reservoir  behavior  over  time  was 
merely  extrapolations  from  a  (local)  “production  curves”  or  “well  performance  models”, 
which  describe  cause-effect  relationships  between  production,  fluid  content  (oil,  water, 
gas),  and  pressures  in  producing  and  injection  wells,  over  time,  using  curve 
fitting/regression  [66].  The  introduction  of  PDE  suddenly  provided  insight  and  extended 
the  prediction  power  significantly,  providing  intelligence  to  an  otherwise  a  black  box 
model.  This  addition  of  intelligence  has  had  a  sweeping  effect,  and  was  extended  to 
groundwater  and  multiphase  flows  in  porous  media,  including  geothennal  reservoirs, 
unsaturated  flow  in  soils,  and  contaminant  transport  in  aquifers  (e.g.,  [9]),  while  it  has 
been  further  reinforced  by  a  rapid  development  of  numerical  methods  and  ever-increasing 
computer  power.  The  success  of  the  numerical  model  that  could  explain  and  predict  the 
subsidence  of  Venice  in  the  early  70’s  [84-86]  has  been  used  as  a  live  example  of  the 
ultimate  intelligence  of  this  new  tool.  By  the  early  80’s,  reservoir  engineers  and 
hydrogeologists  have  developed  numerous  numerical  models  (or  simulators),  which,  with 
the  help  of  new  visualization  tools  and  mainframe  computers,  could  both  predict  and 
visualize  the  movement  of  oil,  gas,  water,  and  contaminants  (in  groundwater),  with 
ultimate  confidence  and  optimism  due  to  the  exponential  growth  of  computer  power  and 
the  prospect  of  optimal  management.  However,  before  long,  it  has  become  clear  that  once 
applied  to  geological  fonnations  on  a  scale  where  the  heterogeneity  cannot  be  neglected, 
the  strength  of  the  PDE-based  model  becomes  its  weakness;  this  sensitivity,  overlooked 
and  even  welcomed  initially  (because  sensitivity  to  certain  inputs  and  parameters  is 
consistent  with  the  physics  of  the  phenomenon),  came  to  haunt  the  modelers  later  on;  the 
exaggerated  expectations  have  turned  into  disappointment  and  distrust. 

How  could  this  be  explained?  A  reservoir  simulator  based  on  PDE  requires  accurate 
definitions  of  reservoir  properties  on  assumed  certain  scales,  everywhere  in  the  reservoir, 
in  order  to  reliably  represent  the  flow,  and  predict  reservoir  responses.  Such  near-ideal 
conditions  could  occur  in  cases  where  the  scales  of  heterogeneities  are  much  smaller  than 
the  simulated  domain,  and  given  the  particular  question  being  asked,  such  heterogeneities 
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could  be  lumped  under  unique,  measurable  reservoir  characteristics  that  could  be 
assumed  unifonn  on  a  particular  scale.  While  this  could  be  the  case  in  many  sandy 
reservoirs  and  aquifer,  on  a  certain  scale,  many  more  reservoirs  and  aquifers  exhibit  non¬ 
uniformity  on  all  scales,  and  thereby,  drastically  degrade  the  prediction  capability  of 
PDE-based  models.  This  has  forced  reservoir  modelers  and  hydrogeologists  to  account 
for  heterogeneity  in  any  possible  way.  Due  to  lack  of  spatial  data,  heterogeneity  translates 
to  uncertainty,  and  uncertainty  translates  to  randomness,  and  thus,  the  PDE  become 
random  (or  stochastic)  PDE  [e.g.,  87],  severely  complicating  and  inhibiting  prediction 
capability.  Moreover,  since  the  uncertainty  is  on  all  scales  and  in  all  parameters, 
structural  model  errors  add  severe,  unquanti  liable  uncertainty  to  the  already  uncertain 
solution  [58,  54,  82].  Before  we  reach  this  bleak  conclusion,  let  us  further  explore  current 
treatment  of  heterogeneity/uncertainty  in  reservoir  simulations. 

2.2  Dealing  with  data  gaps  and  uncertainty 

We  recognize  that  knowledge  of  all  reservoir  flow  properties  on  all  scales 
everywhere  in  the  reservoir  is  impossible  even  without  considering  drilling  costs.  Due  to 
the  high  cost  of  drilling,  there  is  typically  only  sparse  infonnation  on  reservoir  behavior, 
while  geophysical  (esp.  seismic)  data  are  by  far  more  abundant.  As  a  separate  discipline, 
reservoir  modelers  have  no  choice  but  to  rely  on  geological  interpretations  and  limited 
observations  of  reservoir  behavior,  while  being  forced  to  average  and  upscale  reservoir 
properties  using  ad-hoc  estimates  and  geostatistical  tools. 

The  prevalent  method  to  account  for  heterogeneity  (of  reservoir  properties)  and  the 
resulting  uncertainty  is  to  treat  all  the  data  or  interpretations  related  to  one  or  two 
dominant  parameters  (typically,  saturated  permeability)  statistically,  i.e.,  transforming 
uncertainty  into  randomness,  typically  under  the  assumptions  of  underlying  joint 
probability  distribution  (PDF)  that  (assuming  ergodicity)  represents  space  and  time  as 
well.  Once  a  certain  parameter  is  considered  random,  the  PDE  solution  becomes  random 
(or  stochastic)  as  well,  and  additional  theoretical  difficulties  emerge  [56-58,  87].  This 
approach  has  been  developed  and  used  for  the  last  five  decades  in  different  areas  of 
science  and  engineering,  including  stochastic  optimization  of  oil  reservoirs  [1,  10-12,  20, 
28,  36,  51,  59,  65-66,  80-81]  and  groundwater  remediation  [2,  3,  14-15,  19,  21-23,  25-27, 
31,  39-42,  49,  68-73,  77,  83],  Despite  remarkable  theoretical  advances  in  this  difficult  yet 
essential  extension  of  the  deterministic  approach,  developments  have  been  limited  to 
simple  geometries  and  far-reaching  simplifications. 

The  use  of  Monte  Carlo  Simulations  (MCS),  where  “equally  probable”  high- 
resolution  worlds  (or  realizations)  are  generated  and  simulated  in  order  to  compute 
ensemble  statistics  has  been  used  extensively  in  the  areas  of  reservoir  simulations  and 
hydrogeology  (e.g.,  [56-58]);  however,  in  practice,  such  a  procedure  results  in  enormous 
(if  not  prohibitive)  computational  burden  for  predictions  alone,  and  becomes  practically 
prohibitive  for  optimization,  unless  far-reaching  simplifications  are  being  made  (e.g., 
[28]).  In  addition  to  these  limitations,  the  traditional  stochastic  approach  suffers  from  the 
following  drawbacks  and  inconsistencies:  (a)  it  cannot  overcome,  nor  assess  the  major 
uncertainty  in  the  model  structure  (which  remain  rigid);  and  (b)  it  leads  to  additional 
(now  statistical)  models  with  new  parameters  that  are  also  uncertain;  (c)  using 
interpretations  of  well  tests  that  assume  homogeneity  on  a  “near  well”  scale  as  the  basis 
for  conditional  (stochastic)  simulations;  (d)  using  a  single  “dominant”  parameter  (on  a 
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single  scale)  as  the  only  random  property  (otherwise,  computations  are  prohibitive  even 
for  limited  cases);  (e)  assuming  a  PDF  based  on  sparse  spatial  data;  (f)  assuming 
deterministic  boundary  conditions  despite  the  significant  uncertainty  in  it;  (g)  the 
inability  to  capture  the  linked  physics  and  chemistry  on  all  relevant  scales  (thereby, 
missing  important  phenomena  such  as  front  instability  (between  displacing  and  displaced 
fluids  during  enhanced  oil  recovery,  where  micro-scale  variations  trigger  and  promote 
fingering  and  bypassing  due  to  capillary  and  viscosity  differences  [e.g.,  35]  and  various 
geochemical  reactions.  Nevertheless,  reservoir  simulations  and  groundwater  modeling  are 
an  important  basis  for  approximations,  correlations,  and  physical  interpretations, 
including  understanding  and  highlighting  of  the  gaps  and  limitations  of  these 
interpretations. 

Further,  repeating  the  MCS  chain  of  simulations  and  optimization  as  soon  as  new 
infonnation  arrives  is  practically  impossible  under  the  current  scheme.  Thus,  despite  the 
powerful  theoretical  framework  and  insight  provided  by  the  stochastic  approach,  this 
approach  is  yet  in  infancy,  and  does  not  extend  beyond  a  certain  definition  of  parameter 
uncertainty.  Indeed,  when  optimization  is  attempted,  e.g.,  for  a  new  well  placement,  the 
computation-intensive  stochastic  approach  becomes  impossible,  while  a  partial  use  of  the 
approach  (e.g.,  using  only  a  few  Monte  Carlo  simulations,  as  in  [28,  12,  71],  not 
accounting  for  uncertainty  in  other  parameters,  in  the  conceptual  models,  and  in  all 
interpretations  and  decisions  along  the  path  in  Figure  1,  leads  to  largely  non-optimal 
decisions.  If  the  reduction  of  intelligence  can  be  measured  by  the  amount  of  error 
between  optimum  and  non-optimal  operation,  such  a  difference  implies  a  significant 
reduction  in  intelligence  gained  by  physically  based  models. 

In  conclusion,  we  recognize  relationships  between  uncertainty,  model  robustness, 
and  intelligence;  an  ideal  PDE-based  model  is  highly  intelligent  in  a  sense  that  it  can 
predict  reservoir  behavior  at  all  points  in  space  and  time;  however,  as  soon  as  the  model 
structure  is  inaccurate  or  model  parameters  are  uncertain,  it  looses  its  intelligence  to  a 
large  degree.  Practically,  this  implies  that  a  sensitive  model  would  be  “intelligent”  as 
long  as  all  necessary  data  exist  and  are  accurate,  but  drastically  looses  its  intelligence 
where  data  are  uncertain,  inaccurate,  or  insufficient;  on  the  other  hand,  a  less  sensitive 
(more  robust)  PDE-based  model  would  also  be  less  intelligent  to  begin  with.  Thus,  we 
seem  to  face  an  optimization  problem:  what  model  would  be  the  optimal  model  for  a 
particular  problem;  or  better,  what  combination  of  models  would  be  optimal  in  terms  of 
data  use,  (maximum)  intelligence,  and  robustness. 

2.3  Current  use  of  Artificial  Intelligence 

Before  we  answer  this  question,  we  should  be  aware  two  other  sets  of  models:  one 
based  on  artificial  intelligence  (AI)  methods  (also  commonly  termed  soft  computing), 
mostly  in  the  context  of  geologic  analysis  and  oil  explorations,  and  one  based  on  a 
statistical  framework,  particularly  geostatistics  and  Bayesian  statistics.  As  to  AI,  tools 
such  as  artificial  neural  networks,  fuzzy  logic,  genetic  algorithms,  and  probabilistic 
reasoning,  have  been  used  in  reservoir  characterization  [75-76],  subsurface  flow  [29,  63- 
64],  and  well  field  development  and  optimization  [1,  10-11,  13,  28-30,  59,  62-64]. 
Consistent  with  the  AI  approach  is  the  excessive  use  of  geostatistics,  such  as  the  search 
for  best  next  well  placement  described  in  [30],  where  the  authors  bypassed  the  simulator 
altogether,  and  used  indicator  kriging,  instead,  to  interpolate  expected  production  and 
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make  corresponding  decisions.  Nevertheless,  one  of  the  important  outcomes  from  these 
developments  is  the  recognition  of  the  need  in  integration  of  methodologies  rather  than 
using  them  in  isolation  [75].  In  particular,  the  need  to  address  the  issues  of  (a)  integrating 
infonnation  from  various  sources  with  varying  degrees  of  uncertainty;  (b)  finding 
relationships  between  measurements  and  reservoir  properties;  (c)  reducing  uncertainty 
and  risk;  and  (d)  using  all  of  these  to  optimize  reservoir  development  and  management,  in 
real  time.  However,  the  progress  in  this  direction  has  been  slow,  and  fragmented  results 
still  dominate  the  field.  The  main  reason  for  this  is  the  need  to  translate  infonnation 
among  the  subsystems  that  constitute  an  oil  reservoir,  from  geophysics  to  geology,  and 
from  geology  to  reservoir  flow  properties,  and  perform  all  of  these  translations  on 
different  scales  of  information,  with  different  geometric  and  stratigraphic  representations. 
These  integration  problems,  and  the  overwhelming  problem  of  uncertainty  due  to  lack  of 
data  in  the  presence  of  inherent  heterogeneity,  have  been  unresolved,  to  date,  with  only 
scattered  use  of  the  various  computational  tools  for  limited  characterization  and 
prediction  purposes;  hence,  leading  to  non-optimal  management  of  oil  reservoirs,  water 
resources,  and  environmental  cleanup  (groundwater  remediation)  operations. 

How  could  these  capable  AI  models  be  combined  with  all  other  models  in  an 
adaptive  framework  that  will  (a)  account  for  all  the  information  (old  and  new,  without 
initial  filtering),  and  (b)  allow  continual  updating  and  improvement  due  to  continual  data 
accumulation.  The  challenge  is,  thus,  to  integrate  various  measurements/data  and  models 
in  a  comprehensive,  flexible,  adaptive  knowledge  representation  that  will  use  all  the 
available  infonnation  for  optimal  decision  making  in  the  most  intelligent  way  possible. 
Before  trying  to  answer  this  question,  let  us  review  the  third  approach  to 
control/optimization  of  subsurface  flow  and  transport  problems.  One  step  in  that  direction 
was  made  by  Rogers  and  co-workers  [63,  64]  who  ‘trained’  a  ANN  by  using  multiple 
deterministic  flow  and  transport  simulations  of  a  complex  aquifer  under  a  pump  &  treat 
operation,  and  later  [29]  for  oil  reservoir  simulation-optimization,  and  then  used  the 
efficient  ANN  as  a  replacement  (“proxy”)  for  the  cumbersome,  slow  simulator.  Due  to 
the  limited  extrapolation  power  of  the  ANN,  many  model  runs  were  needed  for  the 
training  to  cover  the  expected  span  of  possibilities  (in  the  search  space),  to  enable 
optimization  of  pumping  and  injection  schedules.  Although  the  method  used  is 
deterministic,  as  pointed  out  by  the  authors,  it  could,  in  principle  become  stochastic  by 
generating  multiple  realizations  and  running  Monte  Carlo  simulations  (MCS),  which, 
however,  would  result  in  prohibitive  computer  power.  In  other  words,  since  the 
unsupervised  ANN  used  is  relatively  less  intelligent  than  the  PDE  models,  it  was  used 
only  as  a  minor  auxiliary  function.  We  would  like  to  reverse  this  ranking  of  intelligence 
in  a  way  that  will  enable  broader  conceptualization  and  knowledge  representation. 

2.4  Applying  Dynamic  Control 

Dynamic,  stochastic  control  has  been  used  and  further  developed  mainly  for 
groundwater  remediation  purposes,  specifically  for  pump  &  treat  operations,  which  are 
similar  to  oil  production  operations;  while  the  goal  in  oil  production  is  to  maximize 
production  profit  (over  a  certain  period),  the  goal  of  groundwater  remediation  is  to 
maximize  extraction  of  contaminated  water  from  the  aquifer  over  a  reduced  period.  The 
dynamic-stochastic  control  approach  in  this  field  [6-7,  16-18,  24,  32-34,  37-38,  60-61, 
74]  is  an  extension  of  the  more  general  stochastic  control  theories  of  [8,  50,  67,  78-79]. 
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While  under  the  dynamic  control  approach,  inverse  modeling  (i.e.,  updating/calibrating  of 
uncertain  parameters  in  a  simulator  or  in  a  flow  and  transport  model)  is  done  jointly  with 
the  optimization  process  (hence,  dual  control),  while  feedback  control  rules  enable 
changing  of  pumping  rates  (control  variables)  in  response  to  changing  hydraulic  heads  or 
contaminant  concentrations  (state  variables).  The  stochastic  simulator  varies  from 
extended  Kalman  filter  to  PDE,  using  perturbation  methods  and  dividing  the  cost 
function  into  deterministic  and  stochastic  parts,  with  the  goal  of  minimizing  remediation 
or  plume-containment  cost  while  optimizing  both  sampling  and  control  actions. 
Typically,  differential  dynamic  programming  (DDP)  is  used  to  compute  the  deterministic 
control  [6]  while  the  solution  of  the  stochastic  part  of  the  cost  function  is  obtained 
analytically  using  stochastic  control  techniques  applied  to  the  governing  flow  and 
transport  equations  (PDE),  with  challenging  mathematical  derivation  that  requires  a  twice 
differentiable  cost  function.  The  on-line  parameter  estimation  fed  into  the  flow  equation 
enables  updating  of  both  state  variable  estimates  and  state  covariances.  In  terms  of 
formulation  of  the  cost  function,  the  following  highlights  are  worth  mentioning:  (a)  the 
goal  is  to  minimize  the  average  (estimated,  probabilistic)  cost  function;  (b)  the  cost 
function  is  separable  in  stages,  and  according  to  the  dynamic  programming  approach, 
whatever  the  initial  state  and  initial  decision  are,  the  remaining  (future) 
decisions/solutions  should  constitute  an  optimal  solution  based  on  the  current  state;  i.e., 
the  problem  is  reduced  to  finding  a  current  optimal  control  variable,  given  a  cost  function 
over  the  remaining  (future)  periods,  and  given  the  current  information  state  which 
includes  all  relevant  a  priori  knowledge  of  the  system  and  its  history  of  observations  and 
control;  probabilistically,  this  information  state  is  the  conditional  probability  density 
function  of  the  state  at  the  current  period  conditioned  on  all  past  information; 
consequently,  the  cost  function  depends  on  uncertainty,  directly.  The  two  hidden 
elements  in  this  procedure  are:  (a)  the  Bayesian  approach,  and  (b)  learning  (from  past 
experience). 

Similar  works  [24]  emphasize  the  use  of  all  available  infonnation  to  estimate  all 
present  and  future  uncertainties,  solving  the  management  problem  over  the  designated 
control  horizon,  applying  the  optimal  control  action  (pumping  or  injection)  during  the 
current  time  period,  and  repeating  this  process  at  the  next  decision  time,  with  PDE  (flow 
equations)  treated  as  a  dynamical  state-space  system  using  finite  element  and  finite 
difference  techniques,  considering  (both)  transmissivities  and  boundary  conditions 
uncertain,  and  hence,  perturbed  in  a  highly  simplified  aquifer  system,  with  the  goal  of 
minimizing  pumping  (and  treatment)  costs  while  maintaining  hydraulic  heads  that 
guarantee  containment  of  the  contaminant  plume.  The  results  (a)  provided  insight  into 
system  response  under  uncertainty;  (b)  assessed  trade-offs  between  satisfying  goals  and 
minimizing  uncertainty  (based  on  a  simplified  uncertainty  model);  (c)  emphasized  the 
effect  of  management  decisions  at  any  stage  on  model  predictions  in  the  next  step. 
Explicit  optimization  combined  with  sensitivity  analysis  appeared  to  be  an  effective 
management  approach.  Other  works  [37-38,  60-61]  extended  the  methodology  of  optimal 
estimation  and  scheduling  of  aquifer  remediation  under  uncertainty,  by  allowing  more 
complexity  to  be  introduced,  while  performing  real  time  (dynamic)  feedback  from 
measurements,  as  well  as  joint  (on-line)  parameter  estimation  -  optimization  and 
stochastic  optimization.  Subject  to  constraints  and  a  specified  reliability  of  meeting  water 
quality  requirements  for  a  current  period,  the  method  minimizes  the  expected  value  of  the 
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cost  in  the  next  (remaining)  periods.  A  comparison  between  (adaptive)  deterministic 
feedback  control  and  the  stochastic  control  formulated  by  [37]  showed  a  clear  cost 
reduction  using  the  stochastic  control  formulation,  with  increasing  difference  as  the 
uncertainty  increases.  Despite  the  accommodation  of  more  complexity,  and  more  general 
constraints,  dynamic  control  methods  that  rely  on  PDE  models  are  not  yet  suitable  for 
complex  real  world  problems. 

One  of  the  important  insights  that  emerged  in  this  implementation  of  stochastic 
control  is  the  “probing”  and  “caution”  effects  highlighted  by  Bar  Shalom  [8];  the  effect  of 
the  stochastic/perturbation  part  in  the  dual-control  example  [of  Lee  and  Kitanidis]  is  that 
of  sensitivity  analysis  and  system  excitation  (the  “probing”  effect)  followed  by 
measurements  and  gaining  infonnation  about  system  parameters  that  resulted  in  a 
substantial  improvement.  A  paradigm  shift  is  embedded  here:  rather  than  focusing  on 
general  predictive  power  (or  lack  of  it),  the  dynamic  control  approach  anticipates  how  the 
actual  (future)  state  will  deviate  from  estimated  state  currently  in  hand,  and  steers  the 
system  to  mitigate  possible  losses  (the  “caution”  effect).  These  two  effects  (of  probing 
and  anticipation/caution)  imply  yet  another  effect  -  that  of  goal-oriented  learning. 

The  advantage  of  dynamic  control  was  demonstrated  by  [16]  who  used  differential 
dynamic  programming  to  determine  the  benefits  of  time-varying  optimal  groundwater 
pumping  policies,  with  the  goal  to  reduce  groundwater  concentrations  (of  a  contaminant) 
to  acceptable  levels.  They  demonstrated  that  static  pumping  policies  would  cost  45-75% 
more  than  policies  that  allow  time -varying  pumping  rates,  where  the  management  model 
can  “chase”  the  contaminant  plume.  Another  set  of  developments  along  this  line  [17-18, 
74]  made  use  of  the  “transition  function”  (TF)  that  models  (or  transforms)  the  system 
from  one  state  to  the  next  (in  the  groundwater  contamination  case,  the  TF  consists  of  the 
matrices  generated  by  the  finite  element  model  at  each  time  step)  in  order  to  reduce  the 
number  of  iterations  needed  for  convergence  and  overall  computational  time  in  the 
differential  dynamic  programming. 

A  substantial  use  of  the  second  derivatives  of  the  transition  function  in  a  constrained 
differential  dynamic  programming  (DDP  in  a  complete  fonn)  was  made  [74]  with  respect 
to  a  general  case  pump  &  treat  remediation,  including  pumping  scheduling  and  finding 
best  well  location.  In  this  work  [74],  these  derivatives  were  used  to  generate  feedback 
laws  with  the  aid  of  the  penalty  function  method  (which  converts  the  constrained  optimal 
control  problem  to  unconstrained  optimization,  and  consequently,  allows  flexibility  in  the 
response  of  the  feedback  laws  to  violation  of  constraints).  These  feedback  laws  describe 
relationships  between  required  corrections  of  the  control  variables  and  weighted 
deviations  of  observed  states  from  the  predicted  states.  The  goal  was  to  find  the 
relationships  between  the  second  derivatives  of  the  transition  function  and  evolutionary 
feedback  laws,  where  the  latter  relate  deviations  from  (and  hence,  required  corrections  to) 
optimal  pumping  scheduling  and  deviations  of  heads  and  concentrations  (state  variables) 
from  their  anticipated  states,  through  weights  discovered/assigned  to  these  state 
deviations.  The  methods  requires,  as  a  first  step,  to  employ  a  (deterministic)  model  and 
initial  “optimal”  pumping  policy,  which  enables  to  build  the  first  transition  function,  and 
find  relationships  between  control  and  state  deviations.  The  feedback  laws  are  obtained 
by  adjusting  the  relative  weight  assigned  to  each  penalty  function  (corresponding  to  each 
control  variable). 
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If  we  disregard  the  evolutionary  nature  of  the  feedback  laws,  the  simple  linear 
relationships  expressed  by  the  feedback  laws  (between  observed  deviation  and  required 
action)  resembles  the  inverse  of  action-response  functions  used  in  different  works  [39- 
40].  It  is  interesting  to  note  that  while  the  transition  function  is  derived  from  the 
governing  PDE  (flow  and  transport)  model  (which  could  be  viewed  as  an  elaborated 
response  function  model),  the  feedback  law  represents  cause-effect  rules  (much  like  the 
inverse  of  the  transition  function)  that  compensate  for  model  errors,  regardless  of  the 
source  of  the  errors.  It  is  also  interesting  to  note  that  the  evolution  of  the  feedback  laws 
over  time  has  an  element  of  memory  and  learning  (from  past  cause-effect  relationships). 
Results  from  a  simplistic  example  [74]  showed  to  be  robust  and  efficient  in  terms  of 
reducing  cost  (by  4-51%  less  than  optimization  without  using  a  feedback  law)  as  well  as 
required  computer  time,  for  up  to  25%  deviations  from  mean  parameter  values  (i.e., 
uncertainty  up  to  CV  =  0.25). 

Although  exercised  with  only  small  perturbations  (hence,  small  uncertainty)  and 
some  other  limitations,  this  particular  (complete)  DDP  approach  is  the  first 
control/optimization  method  that  frees  itself  not  only  from  the  need  in  a  rigorous,  well 
defined  statistical/uncertainty  model  (with  assumed  PDF,  correlation  structure,  etc.)  but 
also  free  from  both  parameter  errors  and  model  errors,  yet  without  neglecting  uncertainty, 
and  indirectly,  reducing  it,  which  makes  this  work  a  milestone  that  calls  for  continuation. 
Other  works  have  coupled  optimization  with  network  design  (optimal  monitoring  and 
infonnation  extraction  from  new  wells)  [7,  31,  40-42,  68],  where  the  former  [7]  coupled 
sequential  development  of  the  groundwater  withdrawal  management  with  sampling 
strategies,  dynamically,  which  led  to  the  solution  of  the  withdrawal  design  using  a 
closed-loop  stochastic  control  (dual  control)  method  that  includes  anticipation  of  future 
observation  locations;  the  decomposition  of  the  cost  function  into  deterministic  and 
stochastic  parts,  particularly,  the  inclusion  of  uncertainty  in  the  cost  function  leads  to 
trade-off  between  cost  of  new  wells  and  uncertainty  reduction.  The  sampling  network 
design  method  sequentially  selects  new  measurement  locations  based  on  the  combined 
effect  of  the  state  variable  (hydraulic  head)  uncertainty  at  that  location,  and  the  sensitivity 
of  the  cost  function  to  that  uncertainty.  More  specifically,  new  sampling  locations  are 
selected  using  the  Bayesian  approach  (to  condition  new  measurements  on  existing 
information)  and  based  on  the  product  of  the  sensitivity  of  the  stochastic  part  of  the  cost 
function  and  the  modeled  (predicted)  variance  of  the  state-variable  (hydraulic  head)  at 
that  location;  that  is,  the  sensitivity  of  the  cost  function  to  the  head  uncertainty  is 
weighted  by  the  magnitude  of  the  prediction  error  -  and  vice  versa  (the  prediction  error  is 
weighted  by  the  sensitivity  of  the  cost  function  to  this  error).  The  head  uncertainty  is 
evaluated  by  first-order,  second-moment  groundwater  flow  model,  where  the  head 
uncertainty  is  linked  to  uncertainty  in  hydraulic  conductivity,  boundary  conditions, 
recharge,  and  leakage  (all  are  inputs  of  the  PDE). 

3.  Interim  conclusions 

The  theoretical  developments  and  adaptations  of  methods  from  the  different  disciplines 
of  operations  research,  stochastic  control  theories,  and  artificial-intelligence/soft- 
computing  for  management  of  oil  reservoirs  and  groundwater  remediation  have  provided 
insight  into 
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a)  the  effect  of  uncertainty  (even  if  just  in  one  parameter)  on  optimal  management  and 
cost; 

b)  the  inseparability  of  the  various  components  of  optimal  reservoir  management,  such 
as  optimal  scheduling  and  best  new  well  location  for  either  pumping/injection  or  new 
monitoring  wells; 

c)  the  inseparability  between  optimal  management  and  characterization; 

d)  the  relationships  between  parameter  uncertainty,  reliability,  and  risk; 

e)  the  relationships  between  parameter  uncertainty  and  cost; 

f)  the  effect  of  probing  the  system,  system  anticipation,  and  the  “caution”  that  follows; 

g)  the  similarity  between  the  components  of  sensitivity  analyses,  random  perturbations, 
and  response  functions  and  their  “inverse”  -  weighted  feedback  laws; 

h)  the  ability  to  compensate  for  unknown  model  errors  by  determining  appropriate 
weighted  feedback  policies,  particularly  under  dynamic  feedback  control; 

i)  the  hidden  forms  of  memory  and  learning  that  exist  in  some  statistical  models 
(particularly  Bayesian  statistics),  particularly  where  recursive/evolutionary  information 
processing  takes  place,  as  is  the  case  in  some  dynamic  control  systems,  and  particularly 
where  such  processing  results  in  corresponding  feedback; 

j)  the  strength  of  Bayesian  approaches  in  both  estimation  and  uncertainty  reduction. 

The  advantage  of  the  control  approach  is  in  shifting  the  emphasis  from  one  type  of 
intelligence  -  that  of  predictions  of  first  principle  (physically-based)  models  to  goal- 
oriented  system  anticipation  (the  anticipation  of  the  effect  that  a  control  action  would 
have  on  the  goal,  i.e.,  on  the  cost  function),  as  well  as  shifting  sensitivity  analysis  (of 
general  model  predictions)  to  sensitivity  of  the  cost  function  (to  parameter  uncertainty 
and  particularly,  to  state  uncertainty),  which  changes  the  experimental  design  and  overall 
planning.  This  goal-oriented  intelligence  is  less  “ambitious”  than  the  “know-it-all”  first- 
principle  model.  Our  goal  is  to  increase  the  intelligence  of  the  goal-oriented  anticipating 
model  by  combining/integrating  knowledge  and  models  from  all  disciplines  in  a 
multiresolutional  decision  support  system  (MR-DSS  or  MRDS),  e.g.,  [4-5,  43-48]. 

4.  Increasing  intelligence  with  intelligent  control 

Fortunately,  the  area  of  intelligent  control,  particularly,  the  MRDS  has  been 
developing  rapidly  during  the  last  two  decades,  combining  the  advantages  and  eliminating 
the  limitations  of  control  theories,  operations  research,  and  artificial  intelligence.  For 
example,  an  intelligent  control  agent  such  as  MRDS  could  free  the  dynamic  control  from 
its  ultimate  dependence  on  the  rigid  PDE,  and  can  increase  its  learning,  accumulated 
memory,  and  speed  of  convergence  to  optimal  solutions  by  orders  of  magnitude. 
Moreover,  one  of  the  appealing  outcomes  of  the  stochastic  approach  (including  stochastic 
PDE)  -  the  effect  of  conditioning  on  uncertainty  reduction  via  correlations  among 
variables  -  could  be  amplified  significantly  by  extending  the  associations  among 
variables  on  all  relevant  scales  (through  advanced  MR  clustering  methods)  far  beyond 
linear  the  statistical  correlation  used  in  the  traditional  stochastic  approach;  the  Multi- 
Resolutional  (MR)  knowledge  representation  in  MRDS  maximizes  the  information 
hidden  in  interdependencies  among  these  variables  on  all  levels  of  resolution,  independent 
of  any  particular  single-scale  model.  By  maximizing  extraction  of  infonnation,  the  MR 
approach  effectively  reduces  uncertainty  and  overcomes  the  problem  of  lacking  and 
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corrupted  information.  Most  importantly,  by  using  intelligent  control,  specifically,  a  goal- 
oriented  MR  knowledge  representation,  we  could  eliminate  the  dependency  on  PDE 
models,  and  use  them  just  as  interpretations  and  general  gap-filler  in  the  process  of  MR 
rule  building  (the  latter  being  based  on  experiences  and  cause-effect  relationships).  An 
adaptive  MR  knowledge  representation  is  the  only  way  to  integrate  all  the  methods  from 
all  disciplines  -  to  benefit  from  the  advantages  of  the  different  models  and  eliminate  their 
limitations  (particularly  their  rigid  structure);  to  break  through  the  rigid  serial,  model¬ 
building  process  (currently  done  in  segregation  and  isolation)  and  (hence)  to  enable  more 
powerful  use  of  data  and  knowledge  from  all  disciplines;  to  provide  the  highest 
uncertainty  reduction  possible,  and  efficient  global  stochastic  optimal  control  of  complex 
natural  resources  systems  such  as  oil  reservoirs  and  groundwater,  with  the  highest 
intelligence  and  autonomy  possible  for  particular  goals. 

However,  we  are  still  facing  the  challenge  of  perfonning  and  defining  optimal 
integration  between  PDE  models  and  multiresolutional  representations,  and  since  such 
integration  depends  on  model  quality  and  data  quantity  and  quality,  this  is  an  adaptive 
integration  process  as  well.  One  criterion  for  such  “optimal”  integration  would  be 
uncertainty  reduction  (resulting  from  the  integrated  MR  system),  which  brings  up  yet 
another  challenge:  to  define  the  metrics  for  uncertainty  reduction. 
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